2 


Clickers 201: Exploring the Next Levels of Using 
Classroom Response Systems in Science Courses 


Tanya Noel 
York University 


While there is a wealth of information and resources for new users of classroom response systems, 
much is yet to be explored, particularly for experienced instructors hoping to leverage this technology 
further in support of effective learning. In this paper, I highlight recent literature findings, and sug¬ 
gest some areas for future exploration and discussion. 


Introduction 

C lassroom response systems (CRS) or “click¬ 
ers” are remote-control type devices students 
use to answer questions posed by the instructor; a 
computer, receiver, and software capture responses. 
With these systems, instructors can transform large, 
impersonal lecture halls into dynamic environments 
for active learning, instant feedback, and discussion. 
CRS allow instructors to bridge the physical environ¬ 
ment and the virtual, allowing real-time contribu¬ 
tions from all students, and providing opportunities 
for immediate feedback. 

Despite initial skepticism among science edu¬ 
cators, CRS are gaining more acceptance as tools to 
support learning. In fact, until five years ago, I was 
doubtful that the addition of yet another technology 
to my classroom would be beneficial. However, having 
read some compelling reports (Beatty, 2004; Wood, 
2004), I carried out a pilot study in a large (> 90 stu¬ 


dent) microbiology course. Like many adopters, I was 
struck by the change in classroom dynamic, and stu¬ 
dent feedback was very positive. I wasn’t alone; my 
observations mirrored what other implementers were 
reporting (e.g., Crossgrove & Curran, 2008; Preszler, 
Dawe, Shuster, & Shuster, 2007). 

In the past few years, I have attended sev¬ 
eral (and delivered some) of the many workshops/ 
seminars aimed at individuals considering use of 
CRS. There have been a wealth of such introductory 
(“Clicker 101”) sessions offered at various education¬ 
al conferences, as well as teaching and learning sup¬ 
port centres at universities. Similarly, there is a grow¬ 
ing body of resources for individuals seeking general 
guidelines (see the CWSEI/CU-SEI, Clicker Resource 
Guide, 2009; Duncan, 2005), and best practices (e.g., 
Caldwell, 2007); reports recounting individual expe¬ 
riences in different science classes abound. However, 
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there are fewer resources available for experienced 
CRS users seeking to extend use of this technology. 
In this paper, I have highlighted some of the resourc¬ 
es available in science education literature, as well as 
directions for further exploration. 

What Do We Know? 

Many potentially helpful anecdotal reports are 
available for instructors preparing to use clickers in 
the science classroom, and want to know what to 
expect. The value of using CRS to foster improve¬ 
ment in student performance is also something 
many instructors wonder about. A number of pa¬ 
pers indicate benefits in terms of improved atten¬ 
dance and engagement (see Crossgrove & Curran, 
2008; Gauci, Dantas, Williams, & Kemm, 2009; 
Preszler et ah, 2007). Unsurprisingly, given the dif¬ 
ficulties inherent with implementing comparative 
studies, there is less evidence showing clear benefit 
in learning outcomes. (Indeed, there is considerable 
discussion among educators in terms of how to best 
measure that learning has occurred!) Comparative 
studies can prove challenging in terms of ethical¬ 
ly including appropriate controls, and controlling 
variables. Among comprehensive studies of CRS, 
some have demonstrated significant improvement 
in learning outcomes (Freeman et ah, 2007; Mayer 
et ah, 2009). These studies, and others (Morling, 
McAuliffe, Cohen, & DiLorenzo, 2008), have dem¬ 
onstrated increases in student grades with clicker 
use. 

While Morling et al. (2008) has been cited 
(e.g., Campbell & Mayer, 2008; Morgan, 2008) as 
an example of increased grades in association with 
CRS use, the study reports that clicker use in the 
study group was rare/minimal. In fact, the authors 
note that they deliberately avoided using clickers 
with known pedagogical strategies, and specifically 
mention that future studies could investigate the 
incorporation of CRS with such strategies. View¬ 
ing CRS as a tool, akin to a computer, presentation 
software, or learning management system, I find it 
difficult to envision how clickers could affect stu¬ 
dent learning in a meaningful way without being 


used with established teaching methods. Indeed, 
other educators indicate CRS are tools best used in 
support of known strategies that facilitate student 
learning, such as peer teaching (Mazur, 1997). Be¬ 
atty and Gerace (2009) state: “don’t ask what the 
learning gain from CRS use is; ask what pedagogi¬ 
cal approaches a CRS can aid or enable or magnify, 
and what the learning impacts of those various ap¬ 
proaches are” (p. 147). 

Some studies do clearly indicate pedagogical 
approaches that can be used with clickers. For exam¬ 
ple, CRS are well suited to support question-driven 
activities, known to support high-quality learning. 
Beatty, Gerace, Leonar, and Dufresne (2006) pro¬ 
vide guidelines in setting goals for questions - each 
question should have a clear pedagogic purpose for 
content, process, and metacognition. While their ex¬ 
amples are rooted in physics, I find the guidelines 
extremely helpful in constructing biology questions. 
Examples of CRS questions, as well as practical sug¬ 
gestions, are also available from other sources (e.g. 
Duncan, 2005). 

Dangel and Wang (2008) discuss the use of 
CRS in supporting effective teaching practices and 
promoting deep (vs. surface) learning. They note 
that clickers are commonly used to support only a 
few of the “principles for good practice” in teach¬ 
ing described by Chickering and Gamson (1987), 
such as an emphasis on time-on-task, prompt 
feedback, and facilitating student-instructor con¬ 
tact. Similarly, CRS were primarily used to target 
lower cognitive levels (per Anderson & Krathwohl, 
2001, revised from Bloom, 1956). In some cases, 
CRS were reported as supporting other principles 
of good practice (i.e., to develop student coopera¬ 
tion and active learning, and to communicate high 
expectations). It appears possible to use clickers to 
support all the principles for good practice, and ad¬ 
dress higher-order cognitive skills. 

It is apparent that an instructor seeking a 
magic bullet to improve student performance is like¬ 
ly to be disappointed. There is evidence that CRS 
support good teaching/learning approaches, but as 
with any tool, benefits can be expected only when 
the system is used thoughtfully and aligned with the 
pedagogical strategy chosen by the instructor. 
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Where Are We Going? 

From my own experience, what has been reported in 
the literature, and discussions with other instructors, 
there are several areas where further study/commu- 
nication would be beneficial. I’ve described some be¬ 
low, but this is not an exhaustive list - many avenues 
for exploration and future studies exist! 

Question construction, use, sharing 

Beatty et al. (2006) and others (Nicol 2007; Palmer 
& Devitt 2007) have provided guidelines for the de¬ 
velopment of multiple-choice questions that promote 
effective, higher-order learning. Referencing Bloom’s 
Taxonomy, a significant number of multiple-choice 
questions in the Medical College Admission Test 
(MCAT) assess higher-order thinking skills (Zheng, 
Lawhorn, Lumley, & Freeman 2008). This supports 
the observation by Dangel and Wang (2008), sug¬ 
gesting that there is potential for increased use of 
CRS to foster higher-order thinking skills. 

Although students enjoy in-class demonstra¬ 
tions, Crouch, Fagen, Callan, and Mazur (2004) 
have shown that meaningful learning does not typi¬ 
cally occur just by viewing such demonstrations. 
Ffowever, if students are asked to predict the out¬ 
come of the demonstration, learning increases; click¬ 
ers can be used to poll student predictions ahead of 
a demonstration, and used in an instructor-defined 
framework for post-demonstration reflection and 
discussions. Shared examples of useful experiments 
that have accompanying annotations to help instruc¬ 
tors to use such activities in classes would likely be 
well received in most scientific disciplines. 

Specific guidelines for using/developing 
clicker questions in support of peer instruction, case 
studies, and problem-based-learning activities would 
be welcomed, particularly for instructors of large 
classes who may be daunted by the idea of facilitat¬ 
ing these activities without supporting technology. 

Currently, some publishers make questions 
available to adopters of certain textbooks, but qual¬ 
ity of such question banks is variable. (Test banks for 
multiple-choice exams exist, but also vary in quality, 
and not all of these questions are best used as in-class 


clicker questions.) Tested, annotated banks of clicker 
questions for various scientific disciplines and topics 
would be welcomed. 

Quality of learning, metacognition 

CRS may prove valuable in metacognitive exercises, 
helping students develop awareness of their own 
learning. Crowe, Dirks, and Wenderoth (2008) de¬ 
scribe successful attempts to bring such metacogni¬ 
tion into the classroom. These authors developed 
an assessment tool, which both students and faculty 
could use to evaluate biology questions according to 
Bloom’s Taxonomy levels. Students examine what 
questions are asking for: knowledge, comprehension, 
application, etc. Improvements in student study 
skills and metacognition were observed when stu¬ 
dents were trained in, and used, the assessment tool. 
Wider use of this tool, or other metacognitive exer¬ 
cises tied to clicker questions could promote higher 
quality learning. 

The SOLO Taxonomy is another framework 
for evaluating the quality of learning and progres¬ 
sion towards mastery of a discipline (Biggs & Collis, 
1982). Students can be encouraged to move towards 
deeper learning and develop associated study prac¬ 
tices (Biggs, 2001). I believe that we have an oppor¬ 
tunity to encourage students to assess their own levels 
of mastery and awareness of study practices via CRS, 
similar to the “Blooming Biology” tool described by 
Crowe et al. (2008). (Annotating clicker questions 
with the Bloom’s Taxonomy and/or SOLO Taxono¬ 
my levels targeted would be useful for instructors us¬ 
ing discipline-specific question banks.) 

Concept inventories 

Concept inventories are collections of multiple- 
choice questions designed to determine students’ 
conceptual understanding in a particular field. Ques¬ 
tions deal with core concepts and distractors reflect 
common student misconceptions. The Force Con¬ 
cept Inventory’s (Hestencs, Wells, & Swackhamer, 
1992) success in provoking reform in physics edu¬ 
cation has prompted development of other concept 
inventories. The relationship between CRS and con¬ 
cept inventories is two-fold. Questions from concept 
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inventories delivered through CRS could be helpful 
for instructors to monitor student conceptual learn¬ 
ing in class. CRS can also be used to help identify 
troublesome concepts and common misconceptions, 
which could then be used in the development, or re¬ 
finement, of concept inventories. (Item analyses will 
highlight good vs. poor distractors.) 

Emerging uses of clickers...and potential 
issues 

Several intriguing aspects of advanced CRS use have 
been suggested, but are not well-studied: 

1. CRS could be used to identify at-risk stu¬ 
dents, simply based upon late clicker registra¬ 
tion (not simply attendance, nor poor scores 
on clicker questions), allowing instructor in¬ 
tervention at relatively early stages in a course 
(Griff & Matter, 2008). Other strategies for 
identification of at-risk students are pos¬ 
sible, based on student responses/scores, but 
we await further studies on such methods to 
speculate on the best strategy. 

2. Can CRS affect student retention? The shift 
in classroom activities and atmosphere due to 
CRS use could counter some of the tradition¬ 
al issues students perceive with science cours¬ 
es, leading to attrition (Seymour & Hewitt 
1997; Tobias 1990). Caldwell (2007) reports 
a decrease in attrition during a course, but 
it would be interesting to know if CRS may 
contribute to retention in the longer term. 

3. Use of clickers in class could provide oppor¬ 
tunities for students with disabilities to com¬ 
fortably participate in discussion activities, as 
peer anonymity may encourage more authen¬ 
tic sharing (N. Israelite, personal communica¬ 
tion, 2007). There is also a need to explore 
potential issues faced by students with disabil¬ 
ities; strategies and possible accommodations 
should be developed and disseminated so that 
all students can benefit from the pedagogical 
strategies supported by CRS. 

4. CRS allow data collection in ways that had 
not been easily possible in a classroom setting 


until recently. Reports can be generated for 
various purposes (e.g., student scores, ques¬ 
tion item analysis). Software varies between 
different vendors of CRS, so accessing and 
compiling these data differ by product. Some 
standardization of data files/reports and clear 
guidelines for analysis of CRS data could al¬ 
low individual instructors to leverage collect¬ 
ed data in many ways (e.g., improving clicker 
questions, tracking trends across different aca¬ 
demic sessions, etc.) and provide opportuni¬ 
ties for aggregation of data beyond the indi¬ 
vidual course/instructor. 

5. A concern highlighted by the participants of 
the Clickers 201 STLHE 2009 session relates 
to privacy and ethics of CRS use. As men¬ 
tioned, rapid and copious data collection is 
possible, and instructors can link responses 
to individual students (unless “anonymous” 
mode is selected in the software, or students 
exchange clickers temporarily). Public aware¬ 
ness of privacy rights, along with increasing 
use of CRS, may necessitate clear guidelines 
regarding data collection and privacy. 

Conclusion 

It is an exciting time to be using classroom response 
systems in undergraduate science education. While a 
body of literature and many introductory resources 
exist for new users of CRS, additional information 
and support for experienced CRS users would be 
enthusiastically welcomed. The nature of CRS as a 
tool to support effective pedagogical strategies (as 
opposed to use of clickers just to bring another tech¬ 
nology into the room) has not been widely dissemi¬ 
nated, yet is important in understanding key benefits 
and limitations of the technology. Some potential 
areas for further exploration include sharing of field- 
tested clicker question banks, integration of SOLO 
taxonomy levels in question design/implementation, 
and the use of CRS in development of concept in¬ 
ventories. As we move forward with this technology, 
however, we must keep in mind that considerations 
surrounding accessibility, privacy, and data collec- 
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tion through CRS use have yet to be fully explored. 
I, like many others, am optimistic that a community 
of educators using CRS can develop and share infor¬ 
mation that will serve to improve the use of clickers 
in science teaching. 

Acknowledgements 

Thanks go to Tamara Kelly, Logan Donaldson, and 
the reviewers for helpful comments and suggestions. 

References 

Anderson, L.W., & Krathwohl, D.R. (2001). A tax¬ 
onomy for learning, teaching and assessing: A 
revision of Blooms Taxonomy of educational 
objectives. New York: Longman. 

Beatty, I. (2004). Transforming student learning with 
classroom communication systems. EDU- 
CAUSE Center for Applied Research: Research 
Bulletin, 2004(3), 1-13. Retrieved from 
http://www.educause.edu/ir/library/pdf/ 
ERB0403.pdf 

Beatty, I.D., Gerace, W.J., Leonar, W.J., & Dufresne, 
R.J. (2006). Designing effective questions for 
classroom response system teaching. Ameri¬ 
can Journal of Physics, 74(1), 31-39. 

Beatty, I.D. & Gerace, W.J. (2009). Technology- 
enhanced formative assessment: A research- 
based pedagogy for teaching science with 
classroom response technology. Journal of 
Science Education and Technology. 18, 146- 
152. 

Biggs, J.B. (2001). The revised two-factor Study Pro¬ 
cess Questionnaire: R-SPQ-2F, British Journal 
of Educational Psychology, 71, 133-149. 

Biggs, J. & Collis, K. (1982). Evaluating the quality 
of learning: The SOLO taxonomy. New York: 
Academic Press. 

Bloom, B.S. (1956). Taxonomy of educational objec¬ 


tives: The classification of educational goals. 
New York: Longman, Green. 

Caldwell, J.E. (2007). Clickers in the large classroom: 
Current research and best-practice tips. CBE 
Life Science Education, 6, 9-20. 

Campbell, J. & Mayer, R.E. (2008). Questioning as 
an instructional method: Does it affect learn¬ 
ing from lectures. Applied Cognitive Psychol¬ 
ogy, 23, 747-759. 

Carl Wieman Science Education Initiative, Univer¬ 
sity of British Columbia, and the Science 
Education Initiative, University of Colorado. 
[CWSEI/CU-SEI]. (2009). Clicker resource 
guide. Retrieved from http://www.cwsei.ubc. 
ca/resources/fdes/Clicker_guide_CWSEI_ 
CU-SEI.pdf 

Chickering, A.W. & Gamson, Z.F. (1987). Seven 
principles for good practice in undergraduate 
education. AAHE Bulletin, 39(7), 3-7. 

Crossgrove, K. & Curran, K.L. (2008). Using click¬ 
ers in nonmajors- and majors-level biology 
courses: Student opinion, learning, and long¬ 
term retention of course material. CBE Life 
Science Education, 7, 146-154. 

Crouch, C.H., Fagen, A.P., Callan, J.P. & Mazur, E. 
(2004). Classroom demonstrations: Learning 
tools or entertainment? American Journal of 
Physics, 72, 835-838. 

Crowe, A., Dirks, C., & Wenderoth, M.P. (2008). 
Biology in Bloom: Implementing Bloom’s 
Taxonomy to enhance student learning in 
biology. CBE Life Science Education, 7, 368- 
381. 

Dangel, FI.L. & Wang, C.X. (2008). Student re¬ 
sponse systems in higher education: Moving 
beyond linear teaching and surface learning. 
Journal of Educational Technology Develop¬ 
ment and Exchange, 7(1), 93-104. 



14 


Collected Essays on Teaching and Learning Vol. Ill 


Duncan, D. (2005). Clickers in the classroom: How 
to enhance science teaching using classroom re¬ 
sponse systems. San Francisco: Pearson Educa¬ 
tion. 

Freeman, S., O’Connor, E., Parks, J.W., Cunning¬ 
ham, M., Flurley, D., Fiaak, D., et al. (2007). 
Prescribed active learning increases perfor¬ 
mance in introductory biology. CBE Life Sci¬ 
ence Education, 6, 132-139. 

Gaud, S.A., Dantas, A.M., Williams, D.A., & 
Kemm, R.E. (2009). Promoting student- 
centered active learning in lectures with a 
personal response system. Advanced Physi¬ 
ological Education, 33, 60-71. 

Griff E.R., & Matter, S.F. (2008). Early identifica¬ 
tion of at-risk students using a personal re¬ 
sponse system. British Journal of Educational 
Technology, 39, 1124-1130. 

Fiestenes, D., Wells, M., & Swackhamer, G. (1992). 
Force concept inventory. The Physics Teacher, 
30, 141-151. 

Mayer, R.A., Stull, A., DeLeeuw, K., Almeroth, K., 
Bimber, B., Chun, D., et al. (2009). Clickers 
in college classrooms: Fostering learning with 
questioning methods in large lecture classes. 
Contemporary Educational Psychology, 34, 51- 
57. 

Mazur, E. (1997). Peer instruction: A user’s manual. 
Upper Saddle River: Prentice Hall. 

Morgan, R.K. (2008). Exploring the pedagogical ef¬ 
fectiveness of clickers. InSight: A Journal of 
Scholarly Teaching, 3, 31-36. 

Morling, B., McAuliffe, M., Cohen, L., & DiLoren- 
zo, T.M. (2008). Efficacy of personal re¬ 
sponse systems (clickers) in large, introduc¬ 
tory psychology classes. Teaching of Psychology, 
35, 45-50. 


Nicol, D. (2007). E-Assessment by design: Using 
multiple-choice tests to good effect. Journal 
of Further and Higher Education 31, 53-64. 

Palmer, E.J. & Devitt, P.G. (2007). Assessment of 
higher order cognitive skills in undergraduate 
education: Modified essay or multiple choice 
questions? BMC Medical Education, 7, 49. 

Preszler, R.W., Dawe, A., Shuster, C.B., & Shuster, 
M. (2007). Assessment of the effects of stu¬ 
dent response systems on student learning 
and attitudes over a broad range of biology 
courses. CBE Life Science Education, 6, 29- 
41. 

Seymour, E. & Fiewitt, N.M. (1997). Talking about 
leaving: Why undergraduates leave the sciences. 
Boulder: Westview Press. 

Tobias, S. (1990). They’re not dumb, they’re different: 
Stalking the second tier. Tuscon: Research 
Corporation. 

Wood, W.B. (2004). Clickers: A teaching gimmick 
that works. Developmental Cell, 7, 796-798. 

Zheng, A.Y., Lawhorn, J.K., Lumley, T., & Freeman, 
S. (2008). Application of Bloom’s Taxonomy 
debunks the “MCAT Myth.” Science, 319, 
414-415. 

Biography 

Tanya Noel is an Associate Lecturer in the Biology 
Department and a CST Scholar working with the 
Centre for the Support of Teaching at York Univer¬ 
sity. Interests include understanding/improving qual¬ 
ity of learning in science courses, particularly at the 
first-year level. 



